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Abstract. A class of stochastic optimal control problems containing 
optimal stopping of controlled diffusion process is considered. The nu- 
merical solutions of the corresponding normalized Bellman equations are 
investigated. Methods of [T] are adapted. The rate of convergence of 
appropriate finite difference difference schemes is estimated. 



1. Introduction 

Stochastic optimal control and optimal stopping problems have many ap- 
plications in mathematical finance, portfolio optimization, economics and 
statistics (sequential analysis). Optimal stopping problems can be in some 
cases solved analytically [2J. With most problems, one must resort to nu- 
merical approximations of the solutions. One approach is to use controlled 
Markov chains as an approximation to the controlled diffusion process, see 
e.g. (3J. A thorough account of this approach is available in [3]. 

We are interested in the rate of convergence of finite difference approxi- 
mations to the payoff function of optimal control problems when the reward 
and discounting functions may be unbounded in the control parameter. This 
allows us to treat numerically the optimal stopping of controlled diffusion 
processes by randomized stopping, i.e. by transforming the optimal stopping 
into a control problem, see [5j. This leads us to approximating a normalized 
degenerate Bellman equation. 

Until quite recently, there were no results on the rate of convergence of 
finite difference schemes for degenerate Bellman equations. A major break- 
through is achieved in Krylov [6] for Bellman equations with constant co- 
efficients, followed by rate of convergence estimates for Bellman equations 
with variable coefficients in [7j and [8]. The estimate from [8] is improved 
in [9] and [ID] . Finally, Krylov p] (published in [IT]) establishes the rate of 
convergence r 1 / 4 + h 1 / 2 of finite difference schemes to degenerate Bellman 
equations with Lipschitz coefficients, where r and h are the mesh sizes in 
time and space respectively. 

In the present paper we extend this estimate to a class of normalized 
degenerate Bellman equations which contains those for optimal stopping of 
controlled diffusion processes with variable coefficients. Adapting ideas and 
techniques of [TJ we obtain the rate of convergence r 1//4 + /i 1 / 2 , as in [lj. 
The main ingredient of the proof is a gradient estimate for the solution to 
the discrete normalized Bellman PDE. This is an extension of the gradient 
estimate from [lj to our case. After the first version of this article was sent to 
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arxiv, we received [12] where an essentially more general gradient estimate 
is proved. This opens the way to proving the same rate of convergence result 
for normalized Bellman PDEs in more general setting then we present below. 

Rate of converge results for optimal stopping are proved for general con- 
sistent approximation schemes in |I3j. However, the rate r 1//4 + h 1 / 2 is 
obtained only when the diffusion coefficients are independent of the time 
and space variables. For further results on numerical approximations for 
Bellman equations we refer to [13], [E] and |16j . 

The paper is organized as follows. The main result is formulated in the 
next section. In section [3] the existence and uniqueness of the solution to 
finite difference schemes is proved together with a comparison result. The 
main technical result, the gradient estimate of solutions to finite difference 
schemes, is obtained in section HI Some useful analytic properties of payoff 
functions are presented in section [5J Finally, the main result is proved in 
section [6j 

2. The Main Result 

Fix T G [0, oo). Let (Q, .T 7 , P, {Ft)t>o) be a probability space with a right- 
continuous filtration, such that J-q contains all P null sets. Let {wt,J-t) be a 
d! dimensional Wiener martingale. Let A be a separable metric space. For 
every t G [0, T], x G M rf and a £ A we are given a d x d' dimensional matrix 
a a (t,x), a d dimensional vector (3 a (t,x) and real numbers c a (t,x), f a (t,x) 
and g(x). 

Assumption 2.1. a, (3, c, f are Borel functions of (a,t,x). The function g 
is continuous in x. There exist an increasing sequence of subsets A n of A, 
and positive real constants K,K n , and m,m n , such that \J ne ^A n = A and 
for each n G N, a G A n , 

\a a (t, x) - a a (t, y)\ + \p a (t, x) - f3 a (t, y)\ < K n \x - y\, 

\a a (t,x)\ + \(3 a (t,x)\ < K n (l + \x\), (2.1) 

\c a (t,x)\ + \f a (t,x)\ < K n (l + |x|) m ", \g(x)\ < K(l + \x\) m 

for all x G R d and t G [0,T]. 

We say that a G 2t n if a = (at)t>o is a progressively measurable process 
with values in A n . Let 21 = \J neN %i n - Then under Assumption 12.11 it is well 
known that for each s G [0, T], x G M. d and a G 21 there is a unique solution 
{x t : t G [0,T - a]} of 

x t =x+ I a au (s + u,x u )dw u + [ (3 au (s + u,x u )du, (2.2) 
Jo Jo 

denoted by x^' s ' x . For s G [0, T] we use the notation T(T — s) for the set of 
stopping times r < T — s. Define the payoff function to the optimal stopping 
and control problem as 

u>(s,x)= sup sup v a,T (s,x), (2-3) 

a£2l t£T(T— s) 

where 

v a ' T ( s ,x) =e; 
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ft= c ar (s + r,x r )dr, 
Jo 



and j. means expectation of the expression behind it, with x"' s ' x in place 
of Xt everywhere. It is worth noticing that for 

w n (s,x) := sup sup v a ' T (s,x) 

we have w n (s,x) | w(s,x) as n — > oo. By Theorem 3.1.8 in [5j, w n (s,x) 
is bounded from above and below. Hence w(s,x) is bounded from below. 
However it can be equal to +oo. 

Let 9\ n contain all progressively measurable, locally integrable processes 
r = (rt)tt>o) taking values in [0, n] such that J °° r t dt = oo. Let 9\ = 

We will state without proof a result about randomized stopping. The 
proof (taking into account the possibility that g is a function of (t,x), con- 
tinuous in (t, x)) can be found in [T7]. If A = A n , K = K n , m = m n for 
n > 1 then the following theorem is known from [5] (see Exercise 3.4.12, via 
Lemma 3.4.3(b) and Lemma 3.4.5(c)). 

Theorem 2.2. Let Assumption{2j\ hold. Then for all (s,x) G [0, T] x R d , 
either both w(s,x) and 

supsupE^j f [/ Qt (s + t,x t )e~ Vt 

a£2lrg?R I Jo ^2 4) 

+ r^(st)e~ Vt ]e _ ^ r «* 1 rft + 5 (x T _ s )e-^ T -^^ T ~ a 

are finite and equal, or they are both infinite. 

This theorem is the main tool which allows us to treat the problem of 
optimal stopping of a controlled process as just an optimal control prob- 
lem. The aim of this paper is to find the rate of convergence for numerical 
approximations to the following payoff function: 

v(s,x) = sup v a (s,x), (2.5) 

(■T-s 



v a (s,x)=E c s 



(• l — s 

\ f at (s + t,x t )e- Vt dt + g(x T ■^~ T ~ t 
Jo 



(2.6) 



where xt is the solution to (|2.2I) . Notice that (12. 4h can be written in this 
form such that Assumption 12 . 1 1 is satisfied. 

The immediate aim now is to find an approximation scheme for the payoff 
function. Prom [5] we know that under some assumptions (stricter then 
Assumptions 12. If) the payoff function (|2.5f) satisfies the normalized Bellman 
equation 



supm a (« t + L% + f) = on [0,T)x 
u(T,x) = g(x) for x El 

where 

L° u = "'',,"■>...<; + Yl Pi Ux i ~ cC * u > 



(2.7) 
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afj = \/2{a a a a )ij and m a is a non-negative function of the control param- 
eter called the normalizing factor. A crucial property of m a is that m a afp 
m a bf, m a c a and m a f a are bounded as functions a. Hence it is natural 
to approximate v by approximating the solution of the normalized Bellman 
equation. In this paper we show that the finite difference approximation to 
the solution of the normalized Bellman PDE converges to v. Moreover we 
find that the rate of convergence is the same as in [l] . It is worth noticing 
that to show this we don't need any result about the solvability of the Bell- 
man equation. We now describe the approximation scheme. From now on 
let K > 1 be a fixed constant. 

Assumption 2.3. There exist a natural number di, vectors 4 G and 
functions 

al : [0, T) x R d -> R, b% : [0,T) xR^K, Vfc = ±1, . . . ,di 
such that 4 = £_ k , |4| <K,a% = a%, bf > for k = ±1, . . . , d\ and 

= ^(t,x)44, P?(t,x) = b%(t,x)f k (2.8) 

for any a G A and (i, x) G [0, T) x R d . 

This might appear to be a restrictive assumption. Actually, all operators 
L a which admit monotone finite difference approximations satisfy this. See 
[TS] . Fix r > 0, /i > and define the grid 

M T := {(t, x) G [0, T]xR d : (t, x) = ((jr) A T, h(i x £ x + ... + i dl £ dl )), 

j G {0} U N, i k G Z, fc = ±1, . . . , ±di}. 

Let r^(i) := r for t < T — r and vr(t) := T — t for t > T — r. So 
i + 7r(t) = (t + r) A T. Let Q be a non-empty subset of 

M T :=M T n (\0,T) x R d ) . 

Let T r n(t,x) = u(t + r T (t), x), T h / k u(t,x) = u(t, x + h k £ k ), 

u(t + Tr(t), x) — u(t, x) 



5 T u(t, x) 



6 hkA u(t, x) = and A hk/k u = - 6 hkA S hht -i h u. 

rik 

Let 6q ; £ := 0. Let a£ := (1/2)(<t^) 2 . Consider the following finite difference 
problem: 

supm a (5 T u + L? lU + f a ) = on Q, (2.9) 
u = g on M T \ Q, (2.10) 

where 

L h u = Yl a k Ah k 4k u + ^2 b k $h k ,e k u ~ c ^ 

k k 

and m a is a positive function of a G A such that the following conditions 
hold. 
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Assumption 2.4. Let < A < oo. Functions a^, b%, f a , c a > A, g 
are Borel in t and continuous in a G A for each k = ±1, . . . ,d\. For any 
t G [0,T], x,y G R d and a £ A: 

\o%(t,x) - o%{t,y)\ + |&£(i,s) - + \c a (t,x) - c a (t,y)\ 

+m a \f a (t,x)-f a (t,y)\ + \g(x)-g(y)\<K\x-y\, (2.11) 
\o%\ + bl + m a |/°| + m a c° + |#| < K 

and 

l<m Q [l + c a (t,x)]. (2.12) 

Notice that if Assumptions 12.31 and 12.41 are satisfied then (|2.ip is also 
satisfied. On top of Assumptions 12.11 12.31 and 12.41 the following is required 
to obtain the rate of convergence. 

Assumption 2.5. For all a G A, x G R d and t,s G [0, T] 

K(s,a;) -cr a (t,x)| + \b a (s,x) -b a (t,x)\ 

+ \c a (s,x)-c a (t,x)\ < Klt-sl 1 / 2 , 

m a \f a (s,x) - f a (t,x)\ < K\t - si 1 / 2 . 

The following theorem is the main result of the paper. 

Theorem 2.6. Let Assumptions \2.1l \2.3[ \2.J\ and \2.5\ be satisfied. Let v 
be the payoff function A2.5\) . Then $2.9\) - $TTW with Q = M.t has a unique 
bounded solution v Tt h and 

\v - v T ,h\ < iVr(r 1/4 + h 1/2 ) on M T , (2.13) 
where is a constant depending on K,d,d\,T, A. 

We briefly outline how this is proved in Section Following [I] we "shake" 
the finite difference scheme, we smooth the corresponding solution to get 
a supersolution of the normalized Bellman PDE (|2.7|) . Hence we obtain 
the estimate v < v T h + A^(r 1//4 + /i 1 / 2 ) by using a comparison of v with 
supersolutions to the normalized Bellman PDE. To get 

v T ,h <v + ^(r 1 / 4 + 

h 1 / 2 ) we use the same approach with the roles of v and v Tt h interchanged. 
We "shake" the optimal control problem (|2.5p - (|2.6p . again following [TJ. We 
smooth the resulting payoff function to obtain a supersolution of the finite 
difference scheme. We get the estimate by using a comparison theorem 
established for the finite difference scheme. 

An important consequence of Theorem 12.61 is that one gets the same rate 
of convergence for the optimal stopping and control problem. 

Corollary 2.7. Let w be the payoff function for the optimal stopping and 
control problem 112. 3\) . If Assumptions \2.1\ \2.3[ \2.4\ and \2.5\ are satisfied with 
some positve function m a , then 

sup | m a A — - — ) \S T u + L? u - ru + f a + rg] = on Mr, 
a eA,r>0\ l + rj l yj (2.14) 

u = g on Mt \ -Mt, 
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has a unique bounded solution w Ti h and 

\w-w T>h \ < N T (T lfi + h 1/2 ) on M T , 
where Nt is a constant depending on K, d, d\, T, A. 

Proof. First we use Theorem 12.21 to see that w(s,x) given by (|2,3p is equal 
to (|2.4p . Consider a metric space A := A x [0, oo), with any metric defining 
the product topology. Let A n = A n x [0, n]. Then A = \J neN A n . Let 2l n 
denote the set of progressively measurable processes taking values in A n . 
Clearly 

cT-s 



w(s, x) = sup E" 5 

ag2t 



I S f at {8 + t,xt)e- ( "dt + g(xT-.)e 



■<Pt- 



where 

2l=M2l n , r:=f a + rg, c a = r + c a , (p t = f ^ (u, x u )du 

and xt is the solution to (j2.2f) . Notice that we are still using the same a c 
(3 a . We can check that Assumptions EH E2 El and [23] hold with A, 

1 



= m^' r ) := m a A 



1 + r' 

c Q and f a in place of A, m a , f a and c a respectively. Finally we apply 
Theorem 12.61 to w and the scheme ()2.14p . □ 

Corollary 2.8. The system \2.1J$ is equivalent to: 

supm" [S T u + Lfr u + f a ] < 0, g-u = on Mr, 

supm" [5 T u + L%u + f a ] = 0, g - u < on Mt ( 2 -15) 

and u = g on Mt\Mt- 

Proof. Let e = in (|2.14|) and take the supremum over e € [0, 1] . Hence 
(|2,14p can be rewritten as 



sup 

£G[0,1] 



e sup m a {5 T u + L%u + f a ) + (1 - s)(g - u) 



on Mt, 
u = g on Mt\Mt- 



Notice that the supremum over e is achieved by either taking e = or e = 1. 
Hence it can be seen that this is equivalent to (I2.15|) . 

□ 

3. On the solutions of the finite difference scheme 
First we give two simple examples which justify the condition (I2.12p . 
Examples 3.1. Consider A = [0, oo), m a = (1 + a) -1 and the equation 

sup m a (5 T u) = on Mt 

with the terminal condition u = 1 on Mt \ Mt- If u '■ Mt — > K is any 
non-increasing function in i, then m a 5 T u < 0. Hence, letting a — > oo, we 
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see that u satisfies the equation. Consequently the solution to the above 
problem is not unique. 

Let f a = 1 + a. Consider now the equation 

sup m a {5 T u + f a ) = sup m a 8 T u + 1 = on Mt- 

aeA a£A 

If u is a solution then we have m a 8 T u < 0. Hence sup QG ^ m a 8 T u = 0, 
which contradicts the equation. Thus the above equation has no solution. 

Let Assumption 12.41 be satisfied throughout the remainder of this section. 

Lemma 3.2. There is a unique bounded solution of the finite difference 
problem (2. $) - $2.lty) on Mt- 

Proof. Let 7 = (0, 1) and define £ recursively as follows: £(T) = 1, = 
7~ 1 £(t + Tr(t)) for t < T. Then for any function v 

1 - 7 

8 T (£v) = 7^ S T v — v^v , where v= . 

r 

To solve (I2.9l) - (12.10p for u, we could equivalently solve the following for v, 
with u = £,v: 



M T \Qp9 + lQG[v], (3.1) 



1 

'1 

where for any e > 0, 

G[v] :=v + eC 1 sup m a (5 T u + L^ u + f a ) . 

a 

Then, using the convention that repeated indices indicate summation and 
always summing up before taking the supremum, 

G[v] = sup [p" T T v + Th,l k v + p a v + em a (T T £~ 1 )f a ] , 

a 

with 

v a r = e 1 T- 1 m a > 0, pt = e{2h~ 2 a<l + br x \%)m a > 0, 
p a = 1 _ p « - J2p<* - Evm a - em a c a . 
k 

Notice that 

Pk ~ K£ U 2 h 
and 

evm a + em a c a < et~ 1 K + eK, p T < er^K, 
so for all e smaller than some Eq, p a > 0. Also 

0<J2Pk+P a +Pr = l-em a {u + c a )<l-{l\Ju)em a {l + c a ) 

k 

< 1 - (1 V v)eK~ 1 =: S < 1, 

for some sufficiently small e > 0. Since the difference of supremums is less 
than the supremum of a difference 

|G[f](t, x) — G[w\{t, x)\ < 8 sup \v — w\. 

Mt 

Thus the operator G is a contraction on the space of bounded functions on 
M.T- By Banach's fixed point theorem (|3.ip has a unique bounded solution. 

□ 
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Lemma 3.3. Consider defined on A x Mt and assume that in Q 

supm a / 2 a <oo, /f</ 2 Q . 

a 

Let there be functions u\,U2 defined on Mt and a constant fi > such that 
uie~^ x \, U2e~ f1 ^ are bounded and for some C > 

supm" (5 T ni+L^ Ul + /f + C) 

>supm Q (5 r u 2 + L£u 2 + ./ 2 Q ) on Q (3 ' 2) 

and 

ni < «2 on .Mt \ Q- 
Then there exists a constant r* depending only on K,d\, \x such for r £ 

(0,T*) 

ui<u 2 + TC on M T - (3.3) 
If u\, n 2 are bounded on Mt then liS.cl]) holds for any r. 

To prove this lemma we use the following observation. 

Remark 3.4. Let D e u := u x d\ Dfu := u x i xi tl\ a% = \{a^) 2 . Let D£ 
denote the collection of all n-th order derivatives in x. Consider 

L a (t, x)u(t, x) = Y^ «fc (*, x) Dj k u(t, x) + &£(*, x) D 4 u(t, x). 

k 

For any sufficiently smooth function rj(x), by Taylor's theorem, 

m a \L a 7]{x) -L%T](x)\ < N(h 2 sup |D^n| + /i sup |D^|). 

B/c(x) Bk(i) 

Proo/ o/ LemmaWM Let w = m - u 2 - C(T - t). From (l3T2|) 

sup m" (o~ T u) + L^ w) > 

hence for any e > 

w + e sup m° (5 T w + L^w) > w. 

a 

In the proof of Lemma 13.21 choose e such that p a , p^ are nonnegative. So 
the operator G is a monotone operator. Hence for any ip >w, 

V> + £ sup m Q (5 r + L£ VO > w. (3.4) 

a 

Let 7 G (0,1). Use £ from the proof of Lemma 13.21 Fix 7 € (0,1) later. 
Then 

r 

Let 

iVo = sup — - < 00 and V : = ^oC > C~ r 1 > w, 
M T C C 

where n(x) = cosh(/i|x|) and £ = 77^. Then by Remark 13.41 and since for 

r > 

cosh(ur) smh(ur) , . . 

—72 ^3— ^ cosher) 

we get 

m a L£n(x) < m a L a i 1 {x)+N 1 (h 2 + h) cosh(/i|x| + fiK) < N 2 coBh(p\x\+fiK), 
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with Ni,N2 depending only on fj,, d\,K. Hence 

ft c + k o < cfr-'c, - 1) + iva, * =- ^ "^y < ~. 

Then 

if < V" + e SU P ™> a (&t i> + VO 
<iV C [l + e(r- 1 (7-l)+Ar 3 )] . 
Define ^(7) = r" 1 (7 - 1) + N 3 . Then for r < r* := A^ 1 one has k(0) < 



and k(1) > 0. So one can have 7 such that k < and 1 + £K > 0. By (|3.4jl 
w < ^ + esupm Q (5 T -0 + L^V) <N £(1 + £k) on Q. (3.5) 

If w > then u> + = to. Hence 

Nq = sup — - < sup . (3.6) 

Mt ^ Mt £ 

By the hypothesis w < on A4t \ Q- But k < 0, hence Nq must be and 
so w < on A4t. Finally observe that if \i = 0, A^ = A3 = 0. □ 

Corollary 3.5. If v is the solution of (2.9\ ) - (2.10\) then 

v < 2K 2 (T + 1) +sup\g(x)\. 

X 

Proof. Apply Lemma I3~B1 to v and 

t{t) = K 2 [(T-t) + l]+JVi, 
where 7V"i := sup,,. Then on Q 

SU pm a (d T C + K^ + n 

a 

= sup [-m a (K 2 + c a {K 2 [(T-t) + 1] + N x )] < 0, 

a 

while on Mt \ Q 



€(T) > 8vp\g(x)\ > g{x). 



□ 



Corollary 3.6. Assume that ui,u 2 are functions both satisfying 112. 9\) on Q 
and u\ = g\, 112 = 52 on Mt \ Q- Then 

ui<u 2 + sup(gi -52)+- 

x 

Proof. Apply the comparison principle to U\ and U2 '■= sup 2 .(gi — 92)+- d 

Corollary 3.7. Fix (sq,xq) & Mt- Let v be a solution of \2. 9\) - (2.10\) . 

Define 

\v(s ,x) - v(s ,x )\ 
v = sup j : . 

(s ,x)€M T \x — Xq\ 

Then for all (to, xo) G Mt such that to € [sq — 1, sq], 

\v(s ,x ) - v(t ,x )\ < N(u + l)|s - t \L 



10 



Section 3 



Proof. Assume, without loss of generality, that v > 1. Indeed if v < 1, then 
in the proof of the theorem use v = v + 1. If the result holds for u, then 

N(D + 1) < N(v + 1 + 1) < 2N(u + 1). 

Assume that sq > 0. Shifting the time axis, so that to = 0. Then sq < 1. 
Let 

e so-t f or t < Sq, 

1 for t > so. 
Notice that 7 > 1/T. Define 

■0 = 7^ [C + k(s - t)] + -uk + K(s - t) + u(s , aco), 

7 

where £ = rj(x) = \x — xo\ 2 and k is a (large) constant to be chosen later, 
depending on K, d\ and T only. The aim now is to apply Lemma 13.31 to v 
and ip on M So - 

5 T f (t) = -0f , where = t -1 (1 - e" T ) > if -1 (l - e - *). 
With (., .) denoting the inner product in M. d 

L£ t? = 2a£j/ fe | 2 + 2(x - x ) + hl k ) - c a r, <N l {l + \x- x \) - c a rj. 
Hence 

St C + K C < [^1(1 + |x - xol) - 9\x - x \ 2 ] - c a ( 
< A 2 (l + \x- sc 1) - 0|x - £o| 2 - c a (. 
Since u(so,ss) > —K — sup^ |<?| > —A3 and hence 

m a c> > -m a c a -u + m a c a N 3 . 

7 

Therefore 

< m a Ui> [A 2 (l + \x- xq\) - 6\x - x | 2 -k]+ c a N 3 - c a ^v 

The expression on the right hand side is a quadratic in \x — xq\ with negative 
leading coefficient. It achieves its maximum of 

- k\ + K + m a (c a N 3 - 

when \x — xq\ = ■ Also s o < 1 and so 7 > 1. Since m a (l + c a ) > A -1 , 
m a (-f 2 + c a ) > (1 A 7 2 )A~ 1 . Hence 

^7~ 1 m Q (7 2 + c°) > ki/ 7 " 1 (7 2 A 1)A _1 > A~W. 

Finally, m a c a NT < K. If k is chosen very large (depending on K, T, d only) 
then (13. 7p is non-positive. Hence 

m° (<5^ + L£V + / a ) < 0. 

To apply Lemma l3~3| ^>uon -M so \ A4 S0 must be satisfied. Consider the 
two cases when either j\x — xq\ < 1 or j\x — xq\ > 1. Then 

Tp(s ,x) = V [j\x - X \ 2 + K7" 1 ] + V(S ,X ) 

> v\x - x Q \ + v(s ,xo) > v(sq, x). 
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m a <; 71/ 
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By Lemma 13.31 

v(to, xq) - v(s , x ) < us^^ksq + sI^vk + K sq < N(v + l)s^ 2 . 
To get the estimate from the other side, one would consider 

if> = -71/ [( + K (s - t)] - -uk - K(s - t) + v(s , x Q ) 

7 

and then show that for some choice of k 

supm a (5ip + L%iP + f a )>0 on Q. 

a 

and that ip < v on A4 SQ . □ 
4. Gradient Estimate for the Finite Difference Problem 

The following lemma states some simple properties of the operators 5h,i 
and Aft which will be used in this section. The statements of the following 
lemma are all proved in section 4 of pp. 

Lemma 4.1. Let u and v be functions onM. d , I and xq vectors in M. d and 
h > 0. Then 

$h,i( uv ) = ($h,iu)v + (T h ,iu)5 h) iv = v5 h ,iu + u5 h ,iv + h(5 h ,iu)(5 h ,iv), 

(4.1) 

which can be thought of as the discrete Leibnitz rule. Also 

&h,l (uv) = v A h i u + (S h i u) b h ,iv + (5 h i u)5 h -iv + u A h i v, 

(4.2 

A h ,l( u ) = 2u A h>i u + (8 h> i uf + (6 h -i uf. 

Furthermore if v(xq) < then at xq it holds that 

-Sh,iv <<M*-), ~ A h,lv< A w (u_), (4.3) 

- M"-) < [<M(« + «)-)]- + • ( 4 - 4 ) 

(A ft) fw)_ < [<J ft) _^((^u + «)-)]- + + 

+[<y A) _ < («_)]+ + [<y M («;_)] + . ( - 5) 

I A M u| < I 5 h ,-i((5 h ,i u)-)\ + \ S h ,i((8 h ,-i «)-)|, (4.6) 

I A M u| < I S h ,-i((5 h ,i u)+)\ + I 5fc,i((<yfc,_j u)+)|, (4.7) 

Let Assumption 12.41 hold. Let T" be the smallest integer multiple of r 
which is greater than or equal to T. Choose an arbitrary e G (0, -fT/i] and 
/ G R d . Let h r = h for r = ±1, . . . , ±di and h r = e for r = ±(cZi + 1). Let 

M T (s) ■■= {(t, x + iel) : (t, x) G A4 T , i = 0, ±1, . . .}. 

For a general Q C A4r(e) let 

Q° = {(t, x) : (t + r T (i), s) G Q, (t, x + Mr) G Q, Vr = ±1, . . . , ±(di + 1)}, 

Assumption 4.2. Assume that for any (t, x) G Q® 

\5 hr ,i r b%\<K, m a \5 hrU r\<K, \5 hr j r c a \<K, (4.8) 
\S hrU a a k \< Kv ^ k +h. (4.9) 
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Define a ± = a± = (l/2)(|o| ± a). 

Theorem 4.3. Lei u be a function defined on Atr(e) satisfying \2.9\) on Q. 
If Assumption is satisfied, then there is a constant N > 1, suc/i f/iai, «/ 
f/iere existe some Co > satisfying 

\ + ±(l-e- C0T )> N, (4.10) 

i/ien on Q ; 

|<5 e ,=u«| <iVe Co r / (l + |u| ,Q+ sup (\5 hk/k u\ + \5 £ . lU \ + \te.-M)). (4.11) 
Proof of Theorem \4-3\ Let 



e c °* t < T 



Let (toj^o) G Q be the point where 



is maximized. By definition, for any (i, cc) € Q° we know that 

(i, x + /i r Z r ) G Q. 

Then either 

v r (t,x)<0 or — v r (t, x) = v- r (t, x + h r l r ) < 0. 
In either case, for any (t, x) € Q® 

\v r (t,x)\ < V 1/2 . 

Hence 

1 /9 

Mi := sup |iv| < sup \v r \ + V ' , 

Q.r d e Q,r 

\S £ ,±M < e coT ' sup l^^ul + V 1 ' 2 

S £ Q,r 

on Q. So we only have to estimate V on Q. If (io,xo) belongs to d £ Q, then 
the conclusion of the theorem is trivially true. Thus, we may assume that 
(to,xo) S Qg. Then for any e~o > there exists «o £ ^4 such that at (to,xo), 

m a ° (5 T u + a a k ° A hk/k u + bf 5 hkJk u - c ao u + f a °) +e >0. 

and so for some e G (0, eo] 

m a ° (5 T u + a a k ° A hk/k u + bf 5 hk/k u - c a °u + f a °) + e = 0. (4.12) 

Furthermore (thanks to the fact that (to,xo) E Q®) 

T hr/r [m Q ° (5 T u + a a k ° A hkA u + b a k ° S hhA u - c a °u + f a °)] < 0. (4.13) 

We subtract (|4.12p from (|4.13p and divide by h r to obtain that for each r 

m a ° 5 hrU (5 T u + <° A hk/k u + b a k ° 6 hkA u + T - c^u) - £ < 0. 

For each r, by the discrete Leibnitz rule (|4.ip 

m Q0 (^(r^r) + r 1 [4° A h k A V r + I lr + hr + hr) + S hr ,l r f a °) 

-C 1 S hr ,i r (m ao c ao v)-^<0, (414) 
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where 

hr = {5 hri i r a% )A hkA v, 

hr = h r (6 hrt i r a%°) A hk j k v r , 

hr = (T hrA b a k °)5 hkA v r + (d hrA b a k Q )6 hkA v. 

Notice (first equality by (|4.2[) . second inequality by (|4.3[) . also note that 
there is a tacit summation in A;) that 

> A hk/k J2(v-) 2 = 2v~ A hk/k y- + J2[5 hk/k (v-) 2 + 5 hk j_ k (v~) 2 ] 

r r 

> -2v~ A hkA v r + 2j2($h h A v rf- 

r 

Since a k are nonnegative, 

< v- A hkA v r (4.15) 

and 

I ■■= E <°(^ fc -4 «r ) 2 < A h k ,i k Vr- 

r 

Go back to (|4.14p . multiply by (,v~ and sum up in r to get 

m a ° (^V-Sri^Vr) + |<V &h ht t h V r + \I + V~ [I lr + hr 

+hr +£6 hrilr r°]) - v- 6 hrA (m ao c a °v) -J2t(Vr)t r < 0. 



(4.16) 



By dHHD, m a 5 hrU f% > -K. If v r > 0, then v~ = 0, hence -v~v r = = 
iy~) 2 . On the other hand if v r < 0, then v r = —v~ and so —v~v r = (v~) 2 . 
Hence 

m^v;i 5 hrA f ao ~ v- 5 hr ,iM a °c a °v) 

= m ao v-^5 hrA f ao - m ao v-(5 hr , lr c a °)v - m ao V -(T hrA c a °)v r 

> -e c ° r Kv- - m ao v-\S hr , lr c a °\\v\ + m a °(T hrA c a °)(v~) 2 

> -e CoT 'KM 1 - m a ° KMiM Q + (T hrA m a °c a °)V. 

Use the fact that V attains its maximum at (t ,x ), discrete Leibnitz rule 
and (14. 3D in order to get a lower bound for the term containing 1^. First, 

> ^25 hk/k (v-) 2 = 2v~ 5 hkA v- + ^2h k (5 hk/k v- 



\2 



> 2v r Sh k ,£ k v r > -2v r Sh k ,e k v r . 
Next recall that b k > and | 5h r ,i r ^fcl — ^> an d so 

~v;(T hrA b a k °)5 hkA v r <0, 

hence 

«r > -v-\5 hrA b^°\\5 hkA v\ > -KM 2 . 
Apply discrete Leibnitz rule to the very first term of (|4.16p . Then 

£U~5 T (CV) = [C 1 (t + T)S T V r + VrdrC 1 ] 

= e- c ° T v-5 T v r -V^SrC 1 > -V^SrC 1 
= vUl-e- c ° T ). 
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Using the above estimates we see 

m Q »yi(l - e~ c ° T ) + {T hrA m a °c a °)V + ±m a °o£V A %,4 «r + Am" / 



2' 



+^m ao [Ji r + 7 2r ] - J2&ri: ^ KMi(e c ° T ' +M + m a °Mi). 

Hence 

m «oy !(! _ e -c T) + ^ m «o c «o)y < KM x (e CoT ' + M + m a °Mi) 

+m ao t;-|/ lr | + m ao v;\hr\ - \m a °a a k Q V ; A hkA v r - \m a H + ^v~^;. 

r 

Define 

h :=v-\{b hrU a%°) A hk/k v\ - \ ^2a a k °(5 hk , ek v~) 2 , 

r 

r 

J 3 := h r v-\(5 hrt i r a k °)A hk/k v r \. 
One can rewrite the above inequality as 

m a °V±{l - e~ c ° T ) + (T hr/r m a °c a °)V 

< KM x {e c ° T ' + M + m ao Mi) + m a °(Ji + J 2 ) + £}£v 

r 

So we need to estimate Ji , J 2 • By (|4.6p 

r r 

then because there's tacit summation in k = ±1, ...,±d% and because in 
the second sum we can reverse the order of summation in k 

I A h k ,e k v\ < 2^2\5 hk/k v-\. 

r 

We turn our attention to J\. 

v-\(S hr>lr a^)A hk>ek v\ < MtfKyfff + QAh^vl 

< KM l ^°\A hk/k v\ + KMl 

< KM[ + 2KM 1 ^ Y, 5 h k ,l k V r 

r 

< KMl + Y. mM i + \T.< 0{ - 5 h k i k v-)\ 

r r 

having used Young's inequality. So J\ < NM 2 . Since h r = h or e G (0, Kh], 

J 3 < Khv~Ja%°\ A hk) e h v r \ + Kh 2 v~\A hk)ik v r \. 
Also h? \ Ah kj e k v r \ < 4Mi and in general \a\ = 2a~ + a and so 

J 3 < 2Khv-^{A hkA v r ). + Khv-y/tf°A hk/k v r + NMl 
By (14, 5|) (with summation in r everywhere) 

h(A hk/k v r )- <h\A hkA (v~)\ < \5 hklth (v-)\ + \S hht i_ k (v-)\. 
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Therefore 

1/2 



Jf. < 2Kv r . h\ a? A/,,./, v. 



k -^ hkA v r + 4KM 1 (^24°(5 hk>ik v-) 2 ^ + NM 2 
< NM 2 + 2Kv~h^A hkA v r + \Y, a a k °(5 hk/k v;) 2 , 

r 

J 2 < NM 2 - i«° - AKhyfff)v- A hkA v r . 

Consider K := {k : a£° - AKh^/tf? > 0} . For all k not in K we have a k ° < 
Nh 2 . Also h 2 \A hk/k iv | < 4M X . Hence 

J 2 < ATM? and 
m «oyI(l - e - c ° T ) + (T hr/r m a °c a °)V 

< NM\ (e c ° T ' + M + m a ° M\ \ + J^iV 

r 

We know from (5.5), that Mi < + F 1/2 , where 

/i := sup \v r \ < e CoT ' sup \6 hr j r u\ =: /Z. 
<9 e Q,r 9 £ Q,r 

One also defines 

M D = \u\ 0tQ > e- coT 'M , V = e " 2c » T V 
and so, using Young's inequality 

m ao V±{l - e~ c ° T ) + (T hrA m a °c a °)V 

< N(fL + V^ 2 ) (l + Mo + fi + m^V 1 / 2 ) +Y,&--^ 



< N* (l + M 2 + fl 2 + m a °Vj +J2&r-k- ( 417 ) 

r 

By assumption (5.3), we can find Co such that 

i(l -e~ c ° T ) >l + N*. 
Using this and V < V, (I4.17P becomes 

m^V (1 + T hrA c a ») <N*(l + M 2 + f) + £ 

r 

By our assumptions on m a and c a , m a °(l + T^ r £ r c a ) > ET _1 and so 
V < KN* (l + M 2 + A 2 ) + K ^ 

r 

Noticing that £v~ < e c ° T ' 2M§h-. x 

V < N* (l + M 2 + /x 2 ) + e c ° r NM -^. 
Neither N, nor Mo depend on choice of «o an d so we can let e — > 0. Then 

U < iV* (l + M 2 + // 2 ) . 

□ 
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Theorem 14.31 can now be used to estimate the difference between two 
solutions to the finite difference problem, based on how much the respective 
drift and diffusion coefficients, discounting factor and reward functions differ 
on the domain Q. 

Remark 4.4. For small r > 0, (I4.10|) is satisfied. Indeed, for r _1 > 2N, 
where ./V is the constant arising in (|4.1ip let cq > satisfy e~ c ° T < 1/2. 
Then cq depends on K, d\ only. 

Theorem 4.5. Let 



e = sup l\a%-a%\ + \b a k -b a k \ + \c a -c a \+m<*\P-f c 
M T ,A,k V 

Assume that the functions u, u satisfy \2. 9\) onQ = A4t, with the coefficients 
a, b, c, / and a, b, c, / respectively. Then 

\u-u\< Ne coT 'T'eI, (4.18) 

where 

L (\ s h k A u \ + \ s h k ,e k u\ + \\u(T,x) -u(T,x)\ \ . 

(4.19) 



I := sup I 1 + max( 
(T,x)eM T ^ k 



To prove the theorem we need the following lemma. Recall that a k = 
(1/2)K) 2 . 

Lemma 4.6. Let u and u be functions satisfying \2. 9\) on some Q C Mt 
with the coefficients a, b, c, / and &, b, c, / respectively. Assume that e S (0, h\ 
and for all a E A 

\b%-b%\+m a \f a - / a | + |c Q -c Q | <e, 



\a^-d^\<2e x /a^Ad^ + eh on Q. 

Then on Q 



\u — u\ 



< eNe CoT ' 



1 + Mo,Q + \u\o,Q + sup (| Sh k ,i k u\ + I 6 hk ,i k u\ + e 1 \u- u\) 

k,8Q 



We can deduce this from the gradient estimate by introducing an extra 
coordinate to the grid and by defining a suitable equation on the extended 
grid. 

Proof. We will work with (t,x) = (t,x',x d+1 ) € [0, T] x R d x R. Let 
Q := Q x {0, ±e, . . . , ±me}, 

Q° e := {{t, x', x d+1 ) : (t + r T {t), x', x d+1 ) € Q, (t, x' + h£ k , x d+1 ± e) G Q, 

Vifc = 0,±l,... > ±di} 

8 E Q ■= Q\Q° £ = dQ x {0, ±e, . . . , ±me} UQ°x {me, -me}. 

where m > e^ 1 is a fixed integer. Also let 

~ a a (f j r d+U_{ a a k (t,x') if x^X), 
afclI ' X ' X j ~ \ a%(t,x') if <0 
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and define b? ,c? in a similar way. Let 



kit,*') 



1 _ x d+i- £ 



1 + 



r d+i 



if x d+1 > 0, 
if < 



and define u analogously. Then u satisfies (j2.9|) with d^,b^,c a , f a in place 
of a^?, b^, c a , f a respectively and with the normalizing factor m a , on the 
domain Q. To apply Theorem 14.31 to u one needs to ascertain that Assump- 
tion H2] is satisfied. For r = ±1, . . . , ±di (|4.8|) follows from Assumption 12.41 
Furthermore, for r = ±(c?i + 1) notice that 

if x d+1 > 0, 

8 e/ a%(t,x) = { e- l {al{t,x')-al{t,x')) if x d+1 = 0, 
if x rf+1 < 

holds, while (|4,9p follows 



and similarly for b^,c a . So for 6^ and c 
from: 

e -1 |og(t,a/) - a£(*,a?')| < K x /a%(t,x>,0) + Kh. 

Finally 

m a 5 £/ f a (t,x) = { 



' -m a f a (t,x')/(em) if > 0, 

e- 1 m a (f a (t, x') - f%(t, x')) if = 0, 

m a f a (t,x')/(em) if < 



shows (03]) for / a . By Theorem EE2 for all (i,a/) G .Mr 

e- 1 |«(t,a/)-ti(t,a')| = |ff ei/ fi(t,a;',0)| < iVTe Co ( T+T ) 

1 + No q + SU P ( max I $h k ,t k u\ + \ 6 £ ,e u\ + | 8 £ -e u\ ) 
a, 6 V k J 



(4.20) 



where N is, in particular, independent of m. To complete the proof consider 
d E Q. Either (t, x' , x d+1 ) e dQ x 0, ±e, ... , ±ms. Then 



5 hk/k u(t,x',x d+1 ) 



5 h k ,e k u(t,x') 
5 hk/k u(t,x') 



1 - 
1 + 



E7)l 
y.d+1' 



if x d+1 > 0, 
if x d+1 < 0. 



And so | 5h k ,e k u(t,x',x d+1 )\ < \ 5h k ,e k u(t,x')\ + \ 5h k ,e k u(t,x')\. Also notice 



5 £ „ e u{t,x',x d+l ) 



if / 0, 

e- l (u(t,x')-u(t,x')) if x d+1 = 0. 



if x d+1 ^ 0, 

e _1 (fi(t,a/) -u(t,x')) if = 0. 



Or (t,x',x + ) G Q x {me, —me}. Then 



5^,4 fi(t, x', x d+1 ) 



1 _ £2I^£l if x d+l 



me, 



1 + =£2*1 if x d+1 = -me. 

em J 



Notice that [1 - = 1/m and [1 + = 0. Since m > e -1 , me > 1 

and so for (t, x',x d+1 ) GQ°x {me, —me} 

<5 e .^-u = (L_^tt = 0. 
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Hence 

sup ( max | 8 hki i k u\ + | u\ + \ 5 e _i u\ 



C) s ( 



V k 



< sup (\S hkA u\+2\6 hk/k u\+2£ 1 \u-u\ + \6 hhA u\±). 

k,dQ v ' 

Since the constant N in (|4.20p is independent of m, we can let m — > oo and 
the conclusion of the lemma follows. □ 

Proof of Theorem\]J)\ Let I = (0, . . . , 0, 1) G From our definition of 

e we clearly have 

\b% -b%\ + m a \f a - f a \ + \c a -c a \ <£. 

Assume initially that e £ (0, h]. Then, using that for any a, b > 0, 

\a 2 -b 2 \ = (a + b)\a-b\ = 2(a Ab)\a — b\ + \a — b\ 2 

we get 



K ~a%\< (K| A \6%\)\o% - a%\ + K - a£| 2 < fe^/ag Aa« + ^. 

So we can use 14.61 for Q = .Mt to get 

In — u\ 



< eNe CoT> 



1 + \u\o,q + |w| ,q + sup (| S hk; e k u\ + I S hk) i k u\+e 1 \u - u\) 

k,dQ 



on Mr- Use Corollary 13.51 to estimate the term |it|o,x T + Ho,.M T to obtain 

\u-ii\< eNTe co{T+T) I. 

Drop the initial assumption that e < h. For 9 G [0, 1], let u 9 be the solution 
of 

snpm a (5 T u 9 + a 9 k a A hk/k u 9 + b 9 k a 5 hk/k u 9 - c 9a u 9 + f 9a ) = on M T 

a 

(l-6)u + 6u = u 9 on {(T.i)ejV(t}, 

where 

and of = (l/2)|^ a | 2 . For any 1} 2 G [0, 1], 

Hence if 9\,9 2 satisfy |#i — 6*2 1 ^ < h, then, thanks to the first part of the 
proof (where vP 1 would play the part of u while u 9 " 1 would play the part of 

u), 

\ u 0i -y?2\ < N\e 1 -9 2 \eT'e Co( - T+T h(9 1 ,e 2 ), 

with 



I(9i,9 2 ) := sup ( 1 + max(| 5 hk/k u 9l \ + \ 5 h 4 u 02 |) 
(\0i - e 2 \e)- 1 \u^{T,x) - u 92 (T,x)\^j < 41, 
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since u e = (1 — 9)u + 9u on d £ Q = {(T, x) G A4t}- Subdivide the interval 
[0, 1] into intervals of appropriate length to complete the proof for any e > 
0. ' □ 

5. Some Properties of Payoff Functions 

We assume in the whole section that Assumption 12.11 and the following 
assumption hold. 

Assumption 5.1. There is a function m : A — > (0,1], such that for all 
a G A, t G [0, T), x G R d and y G R d 

m a c a (t,x) +m a > K- 1 , \m a f a {t,x)\<K, (5.1) 
m a \f a (t,x)-f a (t,y)\ <K\x-y\. (5.2) 

Lemma 5.2. There exists a constant N such that: 

(1) For any (s,x) G [0, T] x R d , 

v(s,x) < N(l + (T - s)). 

(2) For any (s,x),(s,y) £ [0,T] xR d , 

\v{s, x) - v{s, y) | < iV(T - s) 2 e JV(T_s) |x - y|. 
Proof. Since m a c a (t, x) + m a > and |m a / a (s,x)| < 

ir( s ,x)i<^ 2 (i+ c a ( s ,x)). 

This and the assumption on the polynomial growth of g together with the 
estimates of moments for solutions to SDEs gives 

K(s,x)|<E^ f T S \r(s + t,x t )\e'^dt + El x \g(xT-s)\e-^ 
Jo 

rT-s 

<K 2 Eg X / (l + c at (s + t,x t ))e-fo cau ( s + u ^)du dt + N 
Jo 

pT—s 

<N(l + (T - s)) + K 2 E / c at {s + t,x t )e-ti cau ( s+u > x ^ du dt 

Jo 

<N(l + (T-s)). 

The right hand side of the estimate is independent of a, hence taking the 
supremum completes the first part of the proof. If solution to a 

stochastic differential equation 



Xf' x = x + / a(s + it, x u )dw u + / (3{s + u, x u )du 
Jo Jo 

where 

\a(t,x) -a(t,y)\ < L\x -y\ and \f3(t, x) - f3(t,y)\ < L\x - y\, 
then, by Theorem 2.5.9 from [5], with N depending on L only, 

E sup \x s f - x s t ' y \ < Ne N ^\x - y\. (5.3) 

t<T-s 
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First get an estimate for a fixed a £ A. 

rT-s 

\v a (s,x)-v a (s,y)\ < E \f at (s + t,x a t >x )\\e-^""' - e~^""" \dt 



Oi,S,X Q[,S,J/ 



+ E T S \f at (s + t,x s t ' x )-f at (s + t,x s t ' y )\e-^' S ' y dt 
Jo 

+ ng(x s /_ s )-g(x s /_ s )\=:h + I 2 +l3. 
Estimating the above integrals separately: 

J 

Since m a c a (t,x) +m a > K^ 1 and \m a f a (s, x)\ < K, 

\f a (s,x)\ < — and c a (s,x) > — 1. 



Therefore, using Lipschitz continuity of c when estimating \^p"' s ' x — ( p't' s ' y \ 

rT-s i t 
h < (T- s)e T - s K 3 E / — sup \x?' s > x - x^ y \e~ Jo K^ du dt 

< (T — s)e T - s K 3 E sup \x a t ^ x - x?> s ' y \ [ " -de~ K^r du . 

t<T-s JO 

Now 

I 2 < f T \m a T l \x? S ' X ~ x?> s ' y \e-v aM 'dt 
Jo 

<E\ sup \xf''' x - xf' 9 ' v \ [ T S K(l + c at (s + t,xt' s ' y )e-^' B ' v dt) 

\t<T-s Jo J 

<[K(t-s)+K]E sup \x?' s > x - xt' s ' Vl 



t<T-s 



and 

h < K{T - s)E sup \x^ x - 

t<T-s 

Hence 

\v a (s,x) -v a (s,y)\ <K 3 (T-s)V T - s )E sup \x?'* - x?'* v \. 

t<T-s 

By ()5,3p the right hand side is independent of a. □ 

Let H T := [0,T) x R d and dH T := {T} x R d . 

Lemma 5.3. Let if) be a smooth function on Ht such that its first order 
partial derivatives in x grow at most polynomially and for all a £ A it 
satisfies 

+ L a %b -c a ip + f a <0 on H T . 

at 

Let v be the payoff function of the stochastic control problem $2.5]) . Then 

v < tp + sup[v — tp]+ on Ht- (5-4) 
8H T 

Hence, clearly, if v < ip on 8Ht, then v <ip on Ht- 
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Proof. For any e > there is a control process at € 21 such that 
r T-s 



v(s, x) < e + E. 

/•T-s 

<E-E Sia! 







-Vt 







9V 



at 



(s + t, x t ) + L at ^{s + i, x t ) - c at i>(s + t, x t ) 



dt 



+ E s , x g(x T -s)e- ipT -°. 

Applying Ito's formula to ip(s + r, x r )e~ lfr on the interval [0, T — s] and 
taking expectation we get 

ip(s,x) =K S;X ijj(T,XT-s)e~' PT - 3 



E, 



T- 



-<Pt 



d_ 

dt 



ip(s + t,x t )dt, 



by noting that the ltd integral has zero expectation, due to moment esti- 
mates for x r , since ip has polynomially growing first order partial derivatives 
in x. Hence 



v(s, x) < e + ip(t, x) + sup [v(T, x) — tp(T, x)]_ 
for any e > 0, which yields (|5.4|) . 



□ 



Lemma 5.4. £e£ ip be a smooth function on Ht such that its first order 
partial derivatives in x grow at most polynomially and there exists ao G A 
such that 

^-iP + L a °iP-c a °ip + f ao >0 on H T . 
at 

Let v be the payoff function of the stochastic control problem 112.5]) . Then 
ip < v + sup[^ — v]+ on Ht- 

Hence, clearly, if ip < v on 8Ht, then ip < v on Ht . 

Proof. For the constant strategy at := ao for all t G [0, T — s] 

i-T-s 

f ao (s + t, x t )e~^dt + g{x T - s )e* T - 



v(s, x) > E 



a 

s,x 
T-s 







0_ 

ot 



+ L ao - c ao 



ip(s + t, x t )e~ Vt dt + g(x T _ s )e- VT - 



By Ito's formula for ip(s + r, x r )e ip " r on the interval [0, T — s] 
ip(s, x) < v(s, x) + sup [ip(T, x) — v(T, x)] + . 



□ 



Corollary 5.5. Assume \2.11\) holds and for all a € A and (t, x) € Ht 

m a \f a \<K and m a c a < K. (5.5) 

Then there exists a constant N such that for (sq,xq) € Ht and (to,xo) £ Ht 
satisfying sq — 1 < to < so one has 

\v(s ,x ) - v(t ,x )\ < N(u + 1) [s - *o|^ s 
where v is the Lipschitz constant from part 2 of Lemma \5.2l 
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Proof. Shift the time axis so that to = 0, so < 1. Let (t,x) G H so := 

[0, so) xR d . Define £(£) = e s °~*. Let ( = ??(^) = |^-^o| 2 an d 7 := Sq" 1 ^ 2 . 
Let 

1 

ip(t, x) = <yv [((t, x) + k(s - t)\ + -uk + w(s , x ), 

7 

where k > 1 is a large positive constant to be chosen later. To apply Lemma 
15.31 we need to show that for all a G A 

m a (Jgl> + L<ty - c> + / a J < on H so 
and i/j(sq,x) > v(sq,x) for any x G M. d . Notice 

^ = "7^-7-, ^(s) = 2(s< - *$), ^J-'/ ^-r 

where <5j is the Kronecker delta. Using (|2.1ip and £ < e T , 

L a ip < -fv2Kd{\ + \x- a? |)£ < "jv2Kd{l + \x- x \)e T . 
Since u is bounded by Nt, for any a £ A and (i, x) G i? So 
-c a (t,x)v(s ,xo) < c a (t,x)N T . 

Hence for any a G A 

-c a ip < -c a ^uK + c a N T on H so . 

Then for any a G A and (t, x) G i? SQ , with N 2 := 2Ke T d and by ([53]) . 

f ^ + L a ip - c a i> + f 
\ot 

< m a j 7 zv [-\x - x \ 2 + N 2 (l + \x-xq\)-k]- c a ^v + c°iV T | + K. 

The right hand side is a quadratic in \x — xq\ with a negative leading coeffi- 
cient and so it achieves its maximum when \x — xq\ = N 2 /2. The maximum 
is 



^2 , , T , N$ 



+ N 2 + 
4 2 



+ c a K^-"u + c a 7V T | + K < M - Lk, 

(5.6) 

where L := K~^v and M depends only on N, d and T. Indeed, so < 1 s ° 
7 > 1. Since m a (l + c a ) > K' 1 on F T , m a ( 7 2 + c a ) > (1 A 7 2 )if~ 1 on H T . 
Hence 

^7" 1 m Q (7 2 + c a ) > ^7 _1 (7 2 A l)^ -1 > K~ x vk = Lk. 

If k is large (depending on K, T, d only) then the right hand side of (15. 6\\ is 
non-positive. Hence 

m a (J^-t/j + L> - c Q V + < 0. 

By Lemma 15.21 

ip(s ,x) = 7^|x - x | 2 + vk^~ x + u(s , x ) 

> 7^|x — xq| 2 + vkj~ — v\x — a?o| + u (so ; ^)- 
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The reader can check that 

. . o i . . 

7^|X — Xq\ — v\x — Xq\ > 0, 

since it is a quadratic in \x — Xq\, its leading coefficient is positive and hence 
its minimum is 4 _1 i^7 _1 + vkj^ 1 — 2~ 1 v^~ l . Hence for all x £ M. d , 

Tp(s ,x) > V(S ,X). 

By Lemma l5T3| ip(to,xo) > v(tQ,xo). But 

v(t , x ) < ip(t ,x ) = ^vksq + 7 _1 i/k + v(s , x ). 
Hence, recalling that 7 = s 1 ^ 2 , 

1/2 1/2 1/2 

v(t ,x ) < v(s ,x ) + s uks + s ' uk < v(s ,x ) + N T (v + 1)V . 
What remains to be shown is that 

1/2 

v(so,x ) < v(t ,x ) + N T (u + l)s ' . 
But that can be achieved by considering 

i> = ~l v [C + K (so ~ t)] - -vk + v(s , x ) 

7 

and applying Lemma 15.41 to ip. □ 

6. Shaking the Coefficients 

The method of shaking the coefficients first introduced in [7] and [8] will 
be used. Recall that v T: h is the solution of (j2.9j) - (j2.10p with Q = Mr- Recall 
that v is the payoff function (|2.5|) . 

Remark 6.1. For (to,xo) not in A4t define v T: h(to,xo) as the solution to 
ra- (l2~TUD on 

M T (t ,x ) = {((t +jT)AT,x + h(i 1 £i + ... + i dl £ dl ) : 

;€NU{0},t fc EZ,k = l,...,d 1 }. 

Let B\ denote the unit ball centered at the origin, in R d . Consider y G 
S C B\ and r e A C (—1,0). Fix e > 0. Let be the unique solution of 



sup m 

a£A,y£S,r£\ 



6 T u(t, x) + L^(t + e 2 r, x + ey)u(t, x) 
+f a {t + e 2 r,x + ey) 



on Mt, 



u(T, x) = sup g(x + ey). 
yes 

(6.1) 

Assumption 12.41 is satisfied by ()6.ip and so the solution exists, is unique and 
has all the other properties proved in section [3l Assume, from now on, that 
r is small enough, so that by Remark 14.41 fl4.10(> holds. 

Lemma 6.2. Let Assumption \2.4\ hold. Then there is a constant N, such 
that 

\v E Tjh -v Tih \<Ne NT Te. 
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Proof. Let the space of controls be A x S x A. Let 

(a%, b a k , c a ,f a )(t, x) := (<r£, b%, c a ,f a )(t + e 2 r, x + ey). 

Then 

sup (k - *k\ + k - m + \c a - n + m a \r - h) < sks. 

Apply Theorem 14.51 to get the conclusion. □ 

The Lipschitz continuity of the solutions to (|2.9p - (l2,10p follows. Consider 
a particular case of (|6.ip . Let A = {0}. Let e = \x — y\. Fix x,y G M. d and 
let S = {e~ 1 (a; — y)}. Say z G S. Then by uniqueness 

v T,h(t^ x ) = v r,h{x - ez) = v Tjh (t, y). 

Hence one obtains: 

Corollary 6.3. Let Assumption \2.4\ hold. Then there is a constant N such 
that for any t G [0, T] and x, y G M d ; 

\v T ,h(t,x)-v Tt h(t,y)\ <NTe NT \x-y\. (6.2) 

Corollary 6.4. Let Assumptions \2.4\ and \2. 51 hold. Then there is a constant 
N such that for any s,t G [0, T] and x G M. d , 

\v rA (t,x) - v T<h (s,x)\ < NTe NT \t - si 1 / 2 . (6.3) 

The proof of this corollary is identical to the proof of (6.8) of Lemma 
6.2 in pQ, if one uses Lemma 13.71 Corollary 16.31 and Theorem 14.51 instead of 
Corollary 3.7, Lemma 6.1 and Theorem 5.6 from PQ, respectively. 

In the continuous case, the shaking will be introduced as follows. Let 
Assumptions [241 and loTTI hold. Consider the separable metric space 

C = Ax{(t,0 G (-1,0) x B 1 }, 

with the metric which comes from taking the sum of the metric for A and 
the metrics which are induced by natural norms on (—1,0) and B\. Extend 
all the functions a,/3,f,c for negative t by cr 7 (t,a;) = cr 7 (0,a;) etc. For a 
fixed e G (0,1), for 7 = (a,r,£) let 

a^{t,x) = a a (t + e 2 T,x + e£) (6.4) 

and similarly for /3, c and /. Let x]' s ' x be the solution of 

ft nt 



s,x 

X f = X + 



' <7 7 (s + u, x u )dw u + / /3 7 (s + u, x u )du. (6.5) 
Jo 

Let C n := A n x {(r, £) G (—1,0) x Bi}, C = {j neN C n . Let <£„ be the spaces 
of admissible control processes defined analogously to 2t n and £ defined 
analogously to 21. Let 

i-T-s 



w^(s,x) = E2 



s.x 



r t (t,x t )e-'*dt + g(xT-s)e 



■<PT- 



(6.6) 



w n (s,x) = sup w 7 (s,i), (6-7) 
w(s,x) = sup w" 1 (s , x) . (6-8) 

7G£ 
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Lemma 6.5. If u> 7 and w are defined by 16. 6\) and \6. 8\) . respectively, then 

\ w l - v <*\ < Ne NT e and \w - v\ < Ne NT e on H T . (6.9) 

Proof. Let x^ ,s ' x and x a ' s,x denote the solutions to (|6.5p and (|2.2p respec- 
tively. By Theorem 2.5.9 of [5] 

E sup \x]> s ' x - x?> s ' x \ 2 <Ne NT snp[\\a a (t + e 2 r,x + eO -a a (t,x)\\ 2 

t<T-s 

+ \(3 a (t + e 2 T,x + eO - (3 a (t,x)\ 2 }. 

(6.10) 

Hence, using Holder inequality 

E sup \xf s ' x - x°' s > x \ < Ne NT e. (6.11) 

t<T-s 

With this in mind the reader could use the same technique used in proving 
the second part of Lemma 15.21 in order to get the desired estimate. □ 

Theorem 6.6. For any e € (0,1] there exists u in C°°([0,T] x M rf ) such 
that 

sup (u t + L a u + f a ) < on H T _ £ 2 (6.12) 

\u-v\< Ne NT e on H T , (6.13) 

I D t u \o,[0,T] xRd + I D x u \o,[0,T] xRd < Ne^e^ on H T , 



D ^lo,[o,T]xRd < Ne NT e 1 on H T . 



(6.14) 



Proof. Recall that A = |J ngN ^4 n - By Theorem 2.1 of [8], there exist a 
smooth function u n defined on Ht such that for all a € A n , 

^- t u n + L a u n + f a <0 on H T _ £ 2. (6.15) 

Let w n be defined by (|6.7p . Notice that / 7 , c 7 , cr 7 and 6 7 in the definition 
all depend on e and xt is defined by (|6.5p . Let C G Co°(( — 1> 0) x^) be non- 
negative with unit integral. Let Q £ {t,x) := e~ d ~ 2 (,(t/e 2 ,x/e). By the proof 
of Theorem 2.1 of [S], u n = w n * ( £ . Let w be defined by (|6.8|) and let u := 
w * ( £ . By Lemma (|5.2|) the functions w,w n are bounded in absolute value 
by a constant independent of n. Then by Lebesgue Dominated Convergence 
Theorem 



d d f 

lim —u n (t,x)= lim — / Q(t - s,x - y)w n (s,y)dsdy 

i^codt n->oo 7(_ 10 ) xBl 

f 9 

= lim w n (s,y)—( £ (t-s,x-y)dsdy 

J(-l,0)xB 1 n ^°° & 



'(-l,0)xSi 
= -u(t,z). 

Hence lim n ^oo Jj^ n = J^u. Similarly linin^oo L a u n = L a u on if-p, for any 
a E A For each fixed a G A, let n — > oo in (I6.15p . Then for any a G A 

^-u + L a u + f a <0 on F T _- 2 , 
at 



which shows (I6.12p . By Lemma 15.21 and Corollary 15.51 

\w{t,x) - w(s,y)\ < Ne NT (\t - s\ 1/2 + \x- y\). 
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Then (16,140 follows from known properties of convolutions. Now we want 
to show (I6.13p . Since u = w * ( £ , \u(s,x) — w(s,x)\ is estimated by the 
right hand side of (|6.13p . So we need only estimate \w(s,x) — v(s,x)\, for 
(s, x) G Ht- To this end we use the fact that difference of supremums is less 
than the supremum of a difference to see that we need only estimate 

[■T-s 



Clearly 

J < K 



\f*{a + t, x1)e-fi - r{s + t, x?)e-v* \dt, 
+E SiX \g(xl_ s )e-^-s - g(a%_ a )e-'ft-\ =: I + J. 



T2e + 2E SX sup \x 



7 _ai 



t<T-s 



I < E,.. T 



+ E S 



T-s 



ris + t^xDWe-^ -e~^\dt 



T-s 



\P*(s + t,xj) - r{s + t,x?)\e-v? dt =: h + I 2 . 



Like in the proof of Lemma f5.2l (using Assumptions l2.4l I2.5l and the definition 

of c 7 ), we get 



h < e s , x 

<K 3 e T 



K\m at )-H 



2e + sup |x 7 — x"\ 

u<t 



■ ™M<Pt ><p?) fa 



2e + E s ^ x sup \x] — x"\ 



t<T-s 

By Assumption 12.41 and the definition of p 



= E SjX [ S \f^( s + t + e 2 r,x2 + ei) - f at (s + t,xT)\e~^ dt 
J o 

pT-s 

< E S)X / \K{m at Y l \ [2e + \x] - xf\] e'^dt 
Jo 



< 2e T K 2 
By Theorem 2.5.9 of [5 



e + E S:X sup \xj — xf\ 

t<T-s 



E 



s .„ sup \xZ - <| < Ne NT E, 

u<T-s 



where x a is a solution of (|2.2|) . while x 7 is a solution of (|6.5|) . Hence, noting 
that the estimate for I + J is independent of a, r and £, we get (|6.13|) . □ 

Lemma 6.7. Let Assumptions [2J[[273[ \2J\ and\EE hold. Then 

Proof. Recall that for e > 0, v e T h is defined as the unique bounded solution 
to (16. ip and by Lemma 16.21 

v% h < v T)h + Ne. 
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Assume, without loss of generality, that h < 1. Let £ > be a C^°([0,T] x 
M. d ) function with a unit integral and support in (—1,0) x B\. For any 
function w defined on (— oo,T) x M. d , for which it makes sense, let 

w( £ \t, x) = \ w(t — e 2 r,x — ey)^(r,y)drdy. 
Jh t 

If the function w(t, x) is not defined for negative t then extend it for t < 
by defining w(t, x) = w(0, x). For any a G A and for all r £ A, y € <S 

<5r - £ 2 r, £ - ey) + L£(i, x)v e T ^ h (t - e 2 r, x - ey) 

+f a (t,x)<0 on H T . 

Multiply this by £(r, y) and integrate over Ht with respect to r and y. Then 

^^y+L^^ + r <0 on H T . 

Use Taylor's Theorem, to get that on H T _ 2e 2: 

I S T " D,<j? | < Ar| D? t#> \ ,n T _ 2e2 =:M X 

+^P'<?lo,H T _ 2e2 =:M 2 . 
With Af := Mi + M 2 , for any a e A on H T _ 2e 2 



T-2e 2 



By Lemma 15, 31 



[B t + L a ] (vfj + (T — t)Af) + /" < 0. 



< u*^ + 2(T - t)M + sup (v - v £ T { 

dH T _ 2e 2 



By Holder continuity in time of v and also : 

sup (v - v £ }^) + < sup(\v(T - e 2 ,x) - g{x)\ + \v £ X\T - e 2 ,x) - g(x)\) 
< AN T e. 

By standard properties of convolutions: 

I D t « T) V|o,H T _ 2e2 + I « r y |o,H T _ 2ea + I D, V T y \0,H T _ 2e2 

< Ne- 3 + Ne' 3 + Ne- 1 . 

Since e = (r + Zi 2 ) 1 / 4 , 

v < w*J? + 4Ae + Are" 3 + Nhe' 1 + A/iV 3 < v £ T f + A(r + /i 2 ) 1/4 . 
Then 

v < v Tih + A(r 1/4 + h 2 ) on H T , 
since \v £ rh — v T ^\ < Ne and \v £ ^ —v e Th \ < Ne. □ 
Lemma 6.8. Let Assumptions \2.1\ \2.3[ \2.J\ and \2.5\ hold. Then 

v r ,h < v + A T (r 1/4 + h 1/2 ) on H T . (6.17) 
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Proof. Let e = (r + /i 2 ) 1//4 . On (T — e 2 ,T] the estimate is trivial consequence 
of the Holder continuity in time of both v and v Ti h- Let S := T — e 2 . It 
remains to prove the estimate on H$- By Theorem 16.61 there is a smooth 
function u defined on [0, T] x R d satisfying 

sup (u t + L%u + f a ) < on H s . (6.18) 

Apply Taylor's Theorem to see that in Ht- t 

\5 T u-B t u\<NT\B 2 t u\ 0tHT =:M 1 

oa I 6 - 19 

m a \L%u-L a u\ < Nh 2 \D*u\o,H T + h\Dlu\ 0i H T =■ M 2 

Let M = M 1 + M 2 . By Theorem ESI 

Mi + M 2 < N T re~ 3 + N T h 2 e~ 3 + A^/ie" 1 < N t (t + h 2 ) 1 ^. 

Since r < 1, e 2 > r. By and dOty 

supm a (5 r ^ + L^ T ^ + r) = 

a 

> sup(5 T u + Ku + / a - M) on ff 5 . 

Let u' = sup Hr \ Hs (v Tj h — u)+ + u. The aim now is to apply Lemma [3731 to 
v Ti h and u'. On the discrete boundary Ht \ Hs> u ' ^ v r,h- By Lemma 13.31 

v Th <u + TM + sup (« Tih -u) + . (6.20) 

By Holder continuity in time of v Tt h and v and by Theorem 16.61 
sup (v T) h — u)+ < sup |u Tj /i — g\ + sup |g — u[ + sup |u — u\ < Nt£- 

Ht\Hs Ht\Hs Ht\Hs Ht\Hs 

By Theorem 16. 6} \v — u\ < Nt£- Hence 

v T ,h <v + N T {£ + r 1/2 + (r + h 2 ) 1 ^) <v + N T (r + h 2 ) 1 / 4 
< v + iV T (T 1/4 + /i 1/2 ) on H s . 

□ 

Acknowledgment. The authors are grateful to Nicolai Krylov in Min- 
nesota for useful information on the subject of this paper. 



References 

[1] N. V. Krylov, "On the rate of convergence of finite-difference approximations for 
Bellman equations with Lipschitz coefficients," arXiv.math, vol. f , no. f , pp. 1-33, 
2004. 

[2] A. N. Shiryaev, Statisticheskii posledovatelnyi analiz. Optimalnye pravila ostanovki. 

Izdat. "Nauka", Moscow, 1976. Second edition, revised. 
[3] J.-L. Menaldi, "Some estimates for finite difference approximations," SI AM J. Control 

Optim., vol. 27, no. 3, pp. 579-607, 1989. 
[4] H. J. Kushner and P. Dupuis, Numerical methods for stochastic control problems 

in continuous time, vol. 24 of Applications of Mathematics (New York). New York: 

Springer- Verlag, second ed., 2001. Stochastic Modelling and Applied Probability. 
[5] N. V. Krylov, Controlled diffusion processes, vol. 14 of Applications of Mathematics. 

New York: Springer- Verlag, 1980. Translated from the Russian by A. B. Aries. 
[6] N. V. Krylov, "On the rate of convergence of finite-difference approximations for 

Bellman's equations," Algebra i Analiz, vol. 9, no. 3, pp. 245-256, 1997. 



29 



References 



[7] N. V. Krylov, "Approximating value functions for controlled degenerate diffusion 
processes by using piece-wise constant policies," Electronic Journal of Probability, 
vol. 4, no. 2, pp. 1-19, 1999. 

[8] N. V. Krylov, "On the rate of convergence of finite-difference approximations for Bell- 
man's equations with variable coefficients," Probab. Theory Related Fields, vol. 117, 
no. 1, pp. 1-16, 2000. 

[9] G. Barles and E. R. Jakobsen, "Error bounds for monotone approximation schemes for 
Hamilton- Jacobi-Bellman equations," SI AM J. Numer. Anal, vol. 43, no. 2, pp. 540- 
558 (electronic), 2005. 

[10] G. Barles and E. R. Jakobsen, "On the convergence rate of approximation schemes 

for Hamilton- Jacobi-Bellman equations," M2AN Math. Model. Numer. Anal, vol. 36, 

no. 1, pp. 33-54, 2002. 
[11] N. V. Krylov, "The rate of convergence of finite-difference approximations for Bellman 

equations with Lipschitz coefficients," Appl. Math. Optim., vol. 52, no. 3, pp. 365-399, 

2005. 

[12] N. V. Krylov, "A priori estimates of smoothness of solutions to difference Bellman 
equations with linear and quasilinear operators," arXiv.math, vol. 1, no. 1, pp. 1-35, 
2006. 

[13] E. R. Jakobsen, "On the rate of convergence of approximation schemes for Bellman 
equations associated with optimal stopping time problems," Math. Models Methods 
Appl. ScL, vol. 13, no. 5, pp. 613-644, 2003. 

[14] E. R. Jakobsen and K. H. Karlsen, "Convergence rates for semi-discrete splitting 
approximations for degenerate parabolic equations with source terms," BIT, vol. 45, 
no. 1, pp. 37-67, 2005. 

[15] E. R. Jakobsen, K. H. Karlsen, and C. La Chioma, "Error estimates for approx- 
imate solutions to Bellman equations associated with controlled jump-diffusions," 
http://www.math.uio.no/eprint/pure_math/2005/pure_2005.html, 2005. 

[16] I. H. Biswas, E. R. Jakobsen, and K. H. Karlsen, "Error estimates for finite difference- 
quadrature schemes for a class of nonlocal Bellman equations with variable diffusion," 
http://www. math. uio. no /eprint /pure-math/ 2006 /pure_2006. html, 2006. 

[17] 1. Gyongy and D. Siska, "On randomized stopping," arXiv.math, no. 1, pp. 1-11, 
2007. 

[18] H. Dong and N. V. Krylov, "On the rate of convergence of finite-difference approxi- 
mations for Bellman equations with constant coefficients," Algebra i Analiz, vol. 17, 
no. 2, pp. 108-132, 2005. 



